Artificial neural networks are computing systems with an architecture based on biological neural networks. Artificial neural networks can be trained, using training data, to learn about how to perform a certain computing task.
A neural network may include a set of processing nodes. Each processing node can perform computations on an input data element to generate an output data element, and the final decision can be generated based on a combination of the output data elements generated by the set of processing nodes. As part of the processing, each processing node can perform a set of arithmetic operations such as floating-point multiplications and additions to generate an intermediate output. Each processing node can also perform post-processing operations on the intermediate output to generate a final output. A neural network may be implemented by an integrated circuit with arithmetic circuitries and data paths to perform the arithmetic operations and post-processing operations, as well as memory devices to store the input data, intermediate outputs and final outputs to support the arithmetic and post-processing operations. Different components of the circuitries and data paths may access the memory devices to read the input data, to store and read the intermediate outputs, and to store the final outputs for the arithmetic operations and post-processing operations.